QUASI-LOCAL MASS AND THE EXISTENCE OF 

HORIZONS 



YUGUANG SHli AND LUEN-FAI TAM^ 

Abstract. In this paper, we obtain lower bounds for the Brown- 
York quasilocal mass and the Bartnik quasilocal mass for compact 
three manifolds with smooth boundaries. As a consequence, we de- 
rive sufhcient conditions for the existence of horizons for a certain 
class of compact manifolds with boundary and some asymptoti- 
cally flat complete manifolds. The method is based on analyzing 
Hawking mass and inverse mean curvature flow. 



1. Introduction 

In this work, we will discuss the relations between different kinds 
of quasi-local mass and use them to derive sufficient conditions for the 
existence of horizons in the time symmetric case. In the time symmetric 
case, a horizon is defined to be a compact minimal surface which is the 
boundary of some open sets (see §3 for more precise definition). 

In 1972, Thorne made the following conjecture, which later became 
known as the hoop conjecture (see [S]): Black holes with horizons form 
when and only when a mass M gets compacted into a region whose 
circumference in every direction is C < AttM . The conjecture is loosely 
formulated . Several concepts such as mass, circumference etc. are not 
clearly defined. Hence, this conjecture allows many different precise 
interpretations. 

In 1983, Schoen and Yau in f^IP, and later Yau in [24J define a kind of 
radius of a bounded region, and derive an upper bound for this radius 
for a region in spacetime without apparent horizons in terms of the 
lower bound of mass density. By this and the study of the obstruction 
to the existence of regular solution to the Jang equation, they obtain 
some important results for the existence of black holes in the spirit of 
the hoop conjecture. 

Date: November, 2005. 

2000 Mathematics Subject Classification. Primary 83C57 ; Secondary 53C44 , 
^Research partially supported by NSF grant of China. 

^Research partially supported by Earmarked Grant of Hong Kong 
#CUHK403005. 



2 



Yuguang Shi and Luen-Fai Tarn 



In the time symmetric case, the method does not work because the 
Jang equation always has a solution. For this case, we will use the in- 
verse mean curvature flow of Huisken and Ilmanen ^21 • We will define 
a quantity m{Q) on a compact three manifold Q with boundary which 
involves Hawking mass of some subsets of Q and some other geometric 
quantities of Q, see definition in ()2.19|) . Then we compare this with 
the Brown- York mass mBy(5f2) of dQ of a simply connected compact 
manifold with nonnegative scalar curvature and with connected smooth 
boundary which has positive Gauss curvature and positive mean cur- 
vature with respect to the outward normal. It is well-known that dQ 
can be isometrically embedded in M.^. The first result is as follows: If Q 
contain no horizons, then mBY{d^l) > m{Q). From this one can prove 
that if m{Q) > 2R where R is the radius of the smallest circumscribed 
ball of dQ in M^, then Q must contains a horizon. In particular, if 
m{Q) > 2diam((9f2), then Q contains a horizon. Examples satisfying 
these conditions are given. 

We will prove that if Q contains a round sphere BE in the sense of 
[7j such that its Hawking mass is larger than mBvid^)-, then VL also 
contains a horizon. However, it is unclear if one can find a round sphere 
satisfying the condition. 

It turns out that the quasi- local mass mB{^) (see the definition in 
§3) introduced by Bartnik for a compact manifold (17, g) with smooth 
boundary with an admissible extension is also bounded below by m(fi). 
It was observed by Walter Simon (see 0) that if (yL^g) has an ad- 
missible extension and suppose (fi, g) is isometrically embedded in an 
asymptotically flat and complete manifold M with nonnegative scalar 
curvature so that the ADM mass mADM{M) of M is less than msi^), 
then M must contains a horizon. Our lower bound for 7/1^(^2) implies 
that if mADM{M) < m{Q), then M must contain a horizon. 

We should emphasis that the quantity m{Q) defined in ()2.19p is 
nontrivial, in the sense that m{Q) > and is zero only if it is locally 
fiat. One can prove that it is actually a domain in M.^ in some cases. 

The basic outline of the paper is as follows. In Section 1, we first 
construct examples to motivate the definition of m{Q). Then we prove 
the positivity of m{Q) (see Theorem 1.1). 

In Section 2, we compare m{Q), mBy(fi) to give sufficient conditions 
for the existence of horizons for compact manifold with boundary as 
mentioned above. We will give another lower bound for the Brown- York 
mass. In particular, we prove that the Hawking mass of the boundary 
is dominated by the Brown- York mass of the domain. 

In Section 3, we prove that m{Q) is bounded above by the Bart- 
nik mass msi^) for Q which has an admissible extension. We will 
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also discuss some properties of Bartnik quasi-local mass including the 
conjecture of Bartnik that ms(fi) is realized by a static admissible 
extension, see j3| for more details of the conjecture. 

The main analytical tool in this paper is the inverse mean curvature 
flow which has been studied by Huisken and Ilmanen [12j. We obtain 
our results by studying the obstruction for the monotonicity of Hawking 
mass under this flow. 



2. Hawking mass of subsets of a domain 

In this section, we will introduce a quantity involving Hawking mass 
of some subsets of a compact three manifold {Q, g) with smooth bound- 
ary which will be used to give a condition for the existence of stable 
minimal spheres on a compact manifold with boundary. All Riemann- 
ian manifolds in this work are assumed to be oriented and connected 
with dimension three. 

To motivate the definition, let us construct some examples. 

Proposition 2.1. There exist asymptotically flat metrics gi, g2 with 
nonnegative scalar curvature on such that gi = g2 outside some 
compact set, {M.^,gi) contains a stable minimal sphere but (M'^, (72) does 
not contain any compact minimal surfaces. 

Recall that an asymptotically fiat (AF) three manifold (M, g) with 
one end is a complete manifold with nonnegative scalar curvature which 
is in L^{M) such that for some compact set K of M, M \ i^' is diffeo- 
morphic to \ Bji{0) for some R> and in the standard coordinates 
in R^, the metric g satisfies: 

(2.1) gij = 5ij + bij 
with 

(2.2) \\bij\\ + r\\dbij\\ + r^WddbijW = 0{r-^) 

where r and d denote the Euclidean distance and standard derivative 
operator on M^. 

Recall also that the ADM mass of M is defined as 



(2.3) rriADM (M) = Mm [ {gu.j - gij,i)iy^dar 

where Sr is the Euclidean sphere, u is the outward unit normal of Sr in 
and the derivatives are taken with respect to the Euclidean metric. 
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Proof of Proposition \2.1\ We first construct metric g2 which contains 
no compact minimal surface such that g2 is Schwarzschild near infinity. 
Let m > be any positive constant, po > ^) and let pi > po- Fix a 
smooth nonincreasing function h on (0, oo) such that h{r) = for any 
r G (0,po), h{r) = — ^ for any r G (pi, oo). Let Cq = 1 + Define a 
function u on as: 



(2.4) u{x) =Co+ I ^dr. 

pi 



for |x| = r. If |x| = r < po, then 

U[x) = Co — 2~ ^ ^ 

J po ^ 

because h<(). If |a;| = r G (po,pi), then 

M X = 1 + r / -^dr > 0. 

2pi Jr 



If |x| = r > pi, then u{x) = 1 + Hence m is smooth and positive. 
Moreover, 

d'^u 2du h' 

Let g2 = u^Qoi where go is the Euclidean metric. Then (M^,5f2) is an 
AF manifold which is Schwarzchild near infinity. 

To prove that (M^,(yf2) contains no compact minimal surfaces, it is 
sufficient to show that each Euclidean sphere with center at the origin 
has positive mean curvature in g2. Let H be the mean curvature of the 
Euclidean sphere Sr = = "r} with respect to g2- Then 



/ X 1 /2 4 9m\ 2 f du\ 

2.5 H=-(- + -j-)= — (n + 2r— 

y r u or J ru'^ \ or J 
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r < then u' = ^ = 0. Since that m > is positive, we have 
H > 0. If r > po, then 

2h(r) m T /i(r) , 

2ru' + u = ^^ + 1 + — + / -\^dT 
r 2pi 

2h(r) ^ m , 1 



r 2pi \pi r 

2h(r) ^ m , , , /I 1 

r 2pi VPi ^ 

(2.6) ^ /I 1 
= l + — + h{r)[ - +- 

2pi VPi 

m m / 1 1 
= ^ + 2;^ ~ 2" + r 

2po 

> 

for some ^ between r and pi, here we have used the fact that h is 
nonincreasing, h> and r > po > m. Hence we also have H > 0. 

In , Miao has constructed a scalar flat AF metric gi on which 
contains a stable minimal sphere and is Schwarzschild at infinity. By 
rescaling, we see that the proposition is true. However, because of later 
application, we will construct gi directly using similar method as in the 
construction of g2. 

Basically, we glue the standard sphere to the Schwarzschild manifold. 
Under the stereographic projection, the metric on the sphere minus a 
point is: 

(2.7) dsl. = \ (dp' + p'da') 

(1 + 4p2)2 

where dp' + p^da'^ is the standard Euclidean metric. On the other hand 
the Schwarzschild metric is given by 



(2.8) rf4, = (l + ^)V' + rW), 

defined on minus the origin, where m > is a constant. We need 
to rescale the metric so that the compact minimal surface {r = l/2m} 
is near p = cxd in the metric ()2.7j) . Namely, let r = ep, then 

Tf) 

(2.9) dsl^ = e\l + —)\dp' + p'da'), 
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For po > 0, define 



(2,10) p>2p„. 

We want to find e > and po so that k can be defined to be nonin 
creasing. Let e = m^/64, po = l/(4m). Then 

, , . 32m2 
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fc(2po) = -64m2. 

Hence there exists mo > such that A;(po) > A;(2po) for all < m < 
mo- For such m we can define /c satisfying ()2.10|) . and is smooth and 
nonincreasing. Next define 



(2.11) U^{x)=h^+ r k{T)dT 

Jo 



if \x\ = p, where bo is chosen such that Um{x) = ^/e(l + m/(2ep)) for 
P > 2po. More precisely, 

r-2po 



m 



(2.12) 



Note that 



r^po 

+ - / A;(r)dr 



m , + 1 - / kMdr 



'■2po 

(2.13) > / k{T)dT > -16m 



Hence 
(2.14) 



m 



PO 



65 8 \ r^Po 



■Umlx) = < ml — 



8 vTO??7 ' ^ !p,Kr)dT, Po<2po; 
+ P>2po, 

for |a;| = p. Hence choosing a smaller mo > 0, we have Um{x) > 
for all X if < m < mo. As before, since k < and k' < 0, we have 
Aum < 0. The metric ds"^ = u^i^dp^ + p'^da^) is an AF metric with 
nonnegative scalar curvature such that on p > 2po, the metric is 

e\l + ^)\dp' + p'da') = (1 + ^)\dr + r'da^) 
zep 2r 
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which is Schwarzschild. Moreover, there is a minimal sphere at 



From the proposition, in order to find a sufficient condition for the 
existence of compact minimal surfaces, we need to know information 
in the interior of the domain. This motivates us to introduce the fol- 
lowing quantity using Hawking mass of some compact surfaces inside 
a domain. 

Let E be an open set in a Riemannian manifold with compact 
boundary, then one can define the mean curvature H of dE in the weak 
sense, see [12]. Recall that the Hawking mass iriH^dE) of dE is defined 
as: 



where \dE\ is the area of dE. In our convention, the standard unit 
sphere in has mean curvature 2 with respect to the unit outward 
normal. 

Let us recall the idea of minimizing hull introduced in ^S]- Let (fi, g) 
be a Riemannian manifold. 

Definition 2.2. Let E he a set in Vt with locally finite perimeter. E 
is said to he a minimizing hull in Q if \d*E (1 K\ < \d*E (1 K\ for any 
set F with locally finite perimeter such that F D E and F \ E CC Q 
and for any compact set K G Q. Here d*E and d*F are the reduced 
houndaries of E and F respectively. E is said to he strictly minimizing 
hull if equality (for all K) implies E P^Vt = F P^Vt a.e. 

Suppose E is an open set of VL such that there is a strictly mini- 
mizing hull in Vt containing E, then define E' to be the intersection 
of all strictly minimizing hulls containing E. Note that E' consists of 
the Lebesque points of the intersection by definition. E' is called the 
strictly minimizing hull of E. Let CC be an open set. Suppose VL 
is compact with smooth boundary which has positive mean curvature 
with respect to the outward normal, then E' exists and E' CC f2. 

Let Vti <Z<Z VL2 <Z VL such that Vti and VL2 have smooth boundaries. 
We need the following lemma from [T3j . 

Lemma 2.3. [Meeks-Yau] With the ahove notations and let d he the 
distance hetween Qi and 80.2 . Let l he the infinmum of the injectivity 




Hence with the metric ds"^ has a horizon and is Schwarzschild at 
infinity. □ 



(2.15) 
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radius of points in {x\ d{x,dQ2) > j}- Let K > be the upper bound 
of the curvature of 0^2 ■ Suppose N is a minimal surface and x & N 
with d{x, 80,2) — |; so that d{x, dN) > |, then 

(2.16) \NnB^{r)\>CK-^ T-^smRrfdr 

Jo 

where r — mm{|, l}. Here C is a positive absolute constant. 
For such Qi, Q2, let 

(2.17) = min | ^-^^ ^, l| 

Let JFqj be the family of precompact connected minimizing hulls with 
boundary in 0,2- Define 

(2.18) m(Qi;Q2)= sup mniE). 
Define 

(2.19) m(Q) = sup Q;ni;Q2"^(^i; ^2) 

where the supremum is taken over all Qi CC ^2 C with smooth 
boundaries. Here and below Vli is always assumed to be nonempty. 

In general, the Hawking mass of a compact surface may be negative. 
However, one can prove that m(Q) > 0. 

Theorem 2.4. Let (fi, g) be a compact manifold with smooth boundary. 
Then Tq,^ 7^ for any fl2 CC ft and m{fl) > 0. IfQ, can be embedded 
in R^, then m(Q) = 0. On the other hand, suppose Q has nonnegative 
scalar curvature and m{Q) = 0, then Q is locally flat. In particular, if 
fl is simply connected, then fl is a domain in M^. 

Proof. Since Q has smooth boundary, by taking a collar of dQ, we can 
embed in a compact manifold ^3 with smooth boundary. Taking a 
double of fls, we may assume that Q is isometrically embedded in a 
compact manifold without boundary. Take a small geodesic disk 
in Q4 \ Q and glue it to the exterior of some compact set of R^, we 
may assume that Q is embedded in a complete noncompact manifold 
M with only one end and such that near infinity of M is isometric to 
the exterior of a compact set in R^. 

Suppose f^2 C ^l. Take a point xo G Q2 and let 3r > be such that 
-Ba:(,(3r) C ^^2, 3r is less than the injectivity radius of xq and dBx^Xp) 
has positive mean curvature for all < p < 3r. We claim that there 
exists < eo < r such that Bx^^p) is a strictly minimizing hull in ^2 
for all < p < eo. Let < p < r and let F be the strictly minimizing 
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hull of Bxq{p) in M. F exists because M is Euclidean near infinity. 
Moreover, there is i? > independent of p such that F C Bp{R) where 
p G M is a fixed point. Suppose Fn{M\BxQ{2r)) ^ 0. Then there is a 
point X e and X G Bp{2R)\B^g{2r). Moreover, S^(r) n5^o(r) = 0. 
Since dF \ -Bxo(^) is minimal surface, see ^^j, by Lemma 1.1 we have 
\dF\ > c > for some constant c > independent of p. Now choose 
r > eo > such that \dBxQ{p)\ < c for all < p < cq. Then for 
< p < eo, the strictly minimizing hull of B^Q^p) must be a subset of 
-Bxo(3r). However, since dB^^^s) has positive mean curvature for all 
< s < 3r, we conclude that F = dB^^lp). This proves the claim. In 
particular, B^g{p) G J-'n^ for all < p < cq and so J^q^ ^ 0. 

Let Hp be the mean curvature of dB^^lp), then it is easy to see that 
Hp = ^ + 0(1) as p ^ and 

Hence we have limp^o "^J/(f^-Bxo(p)) = 0- From this it is easy to see 
that m{Qi; Q2) > for all fli CC Q2 C Q, and so m{Q) > 0. 

It is well know that mH{^) < for any compact surface in M.^, see 
Hence if is a domain in M^, then m{Q) = 0. 

To prove the last two assertions, we first observe that by the defini- 
tion of m{Q), m{fl') < m{Q) if Q' C Q. Now suppose m(r2) = 0, then 
for any xq G fl, we have m{BxQ{p)) = for all p small enough. We 
want to prove fl is fiat near xq using the idea in [12] . Suppose it is not 
fiat near xq, let M be as above. Let po > be such that (i) po < r 
where 3r is the injectivity radius of xq; (ii) dB^a^P) has positive mean 
curvature for all < p < 3r; (iii) for all < p < po, m{Bxo{p)) = and 
BxQ (p) is a strictly minimizing hull in M. Moreover by the proof above 
we may assume that for any open set E C B^Q^po) with boundary, 
then the strictly minimizing hull E' of E in M satisfies E' C Bxq{po)- 

Let Po > p > and let -Ef , < t < 00 be the weak solution of 
inverse mean curvature flow with initial condition i?^g(p) given by a 
locally Lipschitz proper function u^. Such a solution exists by 12j. 
Moreover, for x G ^^^(Sr) 

(2.20) \VuP{x)\<Cid-\x) 

for some constant Ci independent of p and d{x) is the distance between 
X and Xq. By [T21 p. 421-422], we can find q 00, pi such 
that N^^^^ converges for a.e. t G {—00, 00) in to Nt which is a 
solution of inverse mean curvature flow in M \ {xq} given by a locally 
Lipschitz proper function u. Here N^^^. = dE^^^. = d{uP^ < t + Ci} 
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and Nf = dEt = d{u < t}. Moreover, Nt is nearly equal to dB^^^le^) 
as t ^ — oo. Since N^+a is connected for all i, Nt is also connected. 

Let b > — oo be such that -Ef^c^ B^^Xpo) for all t < 6 and for 
all i. Then as in fT^ p. 427], one can show that mniNt) > for all 
t < b. Let us outline the idea. Let t < b he such that Nt^^. converges 
to Nt = dEt in C^. Observe that mH{Nt%^.) > mH{dB^Xpi)) > -^o, 
here 5q is a constant independent of i by the monotonicity formula jT3 
Theorem 5.8] because VL has nonnegative scalar curvature. Since \Nt^^_ \ 
also converges to \Nt\., we have 

(2.21) / <C2 

for some C2 independent of i. Since Nt^^_ converges to Nt in C^, the 
topology of -/V/^+Ci is the same as that Nt^ provided i is large enough. 
By ()2.2H) . [121 Lemma 5.5] and its proof, we have 

(2.22) / \A\^ < C3 

for some constant C3 for all i large enough, where A is the second 
fundamental form of Nt^^_. By lower semicontinuity 

(2.23) / < 00. 

JNt 

Hence the monotonicity formula |12t Theorem 5.8] for A^^^ holds. Note 
that by upper semicontinuity mH{Nt) > for such t. Hence if for 
such a t, mniNt) = 0, then one can prove that mH{Ns) = for a.e. 
s < t. Since there is no compact minimal surfaces in Bx^iSr), one can 
argument as in jT2| that Q is flat near xq. This is a contradiction. 

Now choose to < b such that Nt^^^. converge in to Nt^. Note that 
even though mH^Nto) > and Et^ is a minimizing hull by [T^, Ntg 
may not have boundary. In order to show that m^B^g^p)) > for 
p small, we need some approximation. 

As before, we have 

[ \A\'<oo. 

By (5.19) in |T2], we can find an open set E C B^^^pq) with smooth 
boundary such that mnidE) > 0, dE is connected. Let E' be the 
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strictly minimizing hull of F. Then F' C Bx(Xpo) and 

(2.24) lidpn 

> 

because the mean curvature of dF' is zero on dF' \ OF and the mean 
curvature of dF' is equal to the mean curvature of OF a.e. on dF'CidF. 

Since Br^^lSr) is foliated by geodesic sphere with positive mean cur- 
vature, dF cannot be a minimal surface. Since F is connected, F' 
is connected. Since dF' is C^'^, by Lemma 5.6 in we can find 
G C Bxq{po) with smooth boundary such that G is a strictly minimiz- 
ing hull in M and mH{G) > 0. G can be chosen to be connected because 
F' is connected. Therefore G G ^B^^{po)i we have m^BxQ^Po)) > 0. This 
is a contradiction because m(i?2,Q(po)) = 0. Hence Q is locally flat. If 
Q is simply connected, then is a domain in M.^, see [22, p. 42-44]. 

□ 

3. Sufficient conditions on the existence of compact 

MINIMAL surfaces 

Let {Q, g) be a compact orientable three manifold with smooth bound- 
ary which consists of flnitely many components and with nonnegative 
scalar curvature. In this section, we always assume that the boundary 
of Q has positive Gauss curvature and positive mean curvature with 
respect to the outward normal. Hence each boundary component is 
diffeomorphic to a sphere. We want to obtain a sufficient condition on 
the existence of outermost horizon which is defined as in jl] as follows: 
Let El, . . . , Efc be the components of dQ. For a fixed i, by adding 
points to each component except Sj, we obtain a topological space fli. 
Let Si be the collection of surfaces which are smooth boundaries of 
precompact open sets in Qi containing those points. The boundary dE 
of a set E with dE e Si is called outer minimizing if \dE\ < \dF\ for 
all F D E. A horizon (relative to Sj) is a surface dE e Si with zero 
mean curvature. A horizon is said to be outermost if it is not enclosed 
by another horizon, see [H p. 185]. Suppose dE is a horizon. Let E' 
be its strictly minimizing hull in Qi, which exists because the mean 
curvature of Sj is positive. Then dE' is also a horizon. 

Let {fl,g) and Ej as before. By Weyl embedding theorem, see [T^ . 
each Ej can be isometrically embedded in M.^ and the embedding is 
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unique up to an isometry of M.^. Let H be the mean curvature of Sj 
with respect to the outward normal and let Hq be the mean curvature 
of Sj when embedded in M.^. The Brown- York mass IH] of Sj is 
defined to be 



where da is the volume element on Sj induced by g. 

In this section we will give some lower bounds of the Brown- York 
mass in terms of the Hawking mass of subsets of Q. We will also give 
a sufficient condition on of existence of horizons in Q using the Brown- 
York mass and m{Q) defined in the previous section. The first lower 
bound for the Brown- York mass is the following: 

Theorem 3.1. Let {^l,g) be a compact three manifold with connected 
smooth boundary and with nonnegative scalar curvature. Assume that 
Q is simply connected and suppose dfl has positive Gauss curvature 
and positive mean curvature with respect to the outward normal. Then 



for any connected minimizing hull E in Vt where E GG ^ with C^'^ 
boundary. Moreover, equality holds for some minimizing hull E with 
the above properties if and if Q is a standard ball in and E is a 
standard ball in Q. In particular, mBY{d^^) > mnidQ) and equality 
holds if and only if Q is a standard ball in M^. 

Proof. Let E he a minimizing hull in Q with C^'^ boundary such 
that E GG Q. We want to prove that mBrid^) > mH{dE). Since 
m^y((9f2) > 0, it is sufficient to prove the case when mnidE) > 0. 

Isometrically embed E = dQ in M^. As in one can glue Q to the 
exterior of dQ in to form a manifold (M, h) such that the metric h 
satisfies: 

(i) h\n = g; 

(ii) hM\n is smooth up the boundary; 

(iii) the scalar curvature of /i in M \ is zero; 

(iv) h is Lipshitz near dQ; 

(v) the mean curvatures of dQ with respect to the outward normal 
are the same for metrics inside and outside fl; 

(vi) h is asymptotically fiat; 

(vii) if is the surface consisting points in the exterior to E in 

and with Euclidean distance r from S, then has positive 
mean curvature with respect to h, which is bounded from below 
by +r) for some C > 0. 



(3.1) 




(3.2) 



m BY {dfl) > rriH^dE). 
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Given e > 0, by (i)-(vi) and fHl, there is a smooth AF metric on 
M with nonnegative scalar curvature such that 

(3.3) (1 - e)h,{v, v) < h{v, v) < (l + e)h,{v, v) 
for all tangent vector v on M, and such that 

(3.4) \im m AD M (K) = m ADM (h) 

where niADMih^) and rriADMih) are the ADM masses of {M,hf) and 
(M, h) respectively. 

Let ^ > be given. We can find a connected open set F D E with 
smooth boundary such that F G<Z fl, 
(3.5) 

\dE\-9 < \dF\ < \dE\+9; mnidE) < mnidF) +9; and mnidF) > 0. 

For any e > 0, let F' be the strictly minimizing hull of F in (M, h^). 
F' exists, precompact in M and has C^'^ boundary. Since F is con- 
nected, F' is connected. Moreover M is simply connected because fl is 
simply connected and dQ is homeomorphic to the sphere. Hence dF' 
is connected. 

Choose 5 > be small enough, such that the boundary of Qt for 
all < t < (5 has positive mean curvature in g, where Qt = {x & 
Q\ dg{x,dQ) > t}. Here dg is the distance with respect to g. There is 
<to < 6 such that F CC Qto- Suppose F' \ 7^ 0. Since M\Qs is 
foliated by compact surfaces of positive mean curvature with respect 
to h, if we denote the unit outward normal vector in h of the surfaces 
by z/, we have 

< / divhvdVh 
JF'\nto 



(3.6) 

= / _ h{u,fi) - iF'ndQtolg 
JdF'niM\nto) 

where /i is the unit outward normal of dF' in h. Here divh is the 
divergence with respect to h. Hence we have 



(3.7) \dF'niM\nt,)\,,>\F ndn 



to \g 



Now F' n Qto D E is a set of finite perimeter and E is a minimizing 
hull in {^l,g), we have 



(3.8) 



\d{F'nnj\g>\dE\g. 
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From ()H.7j) and ()3.8|1 we conclude that 

I^F'U = \dF' n (M \ fiiJI, + \dF' n fiij, 
(3.9) > \F'ndQt,\g+\dF'nQt,\g 

= \diF'nn,,)\,>\dE\g. 

Moreover the mean curvature of dF' in is zero on dF' \ dF and 
the mean curvature of dF' and dF are equal a.e. in dF' fl dF. Using 
the inverse mean curvature flow with initial data F', we obtain by the 
Penrose inequality in J2] we have 



(3.10) 



> 



> 



> 







IGtt 










K 




\dF'\ 





IGtt 



dF' 



IGvr V IGtt 




Here, we have use the assumption mH{dE) > in the fifth inequality 
above, and dah^ and dag are the area elements on surfaces induced by 
and g respectively and we have used ()3.5|) . and ()3.9p . Let e — >■ 0, 
and then let 6' — 0, we have niADMih) > mH^dE). By j^I], we have 

(3.11) mBY{dn) > TTiADMih) > mnidE) 

If rriBYid^l) = rriHidE) for some E with the properties in the theorem, 
then rriBYidQ) = rriADMih) by p.lip . Hence f2 is a domain in by 
PTj and rriHidQ) = niBvidn) = 0. By is a standard ball in R^. 

Since mH{dE) = E is also a standard ball in R'^. 
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Conversely, if f2 is a domain in M^, since each sphere has zero Hawk- 
ing mass, and a ball which is small enough is a minimizing hull, we 
have rriBYi^) = mH{dE) = for some small ball. 

To prove the last statement, we observe that Qt is a minimizing hull 
in f2 if it > is small. Hence m5y((9f2) > limt^QmuidQt) = mH{dQ). 
In fact, from the proof above, m^y > limj^o 'mnid^lt) unless f2 is a 
domain in M^. Hence by a result of Willmore |2S1; mBY{d^l) = rriHidQ) 
implies that dQ is a standard sphere so f2 is a standard ball in M^. □ 

In the above theorem, we do not assume that {Q, g) contains no 
horizons. In order to obtain a sufficient condition that {Q, g) contains 
a horizon, we need another estimate of the Brown- York mass. Let 
m{Q) as defined in ()2.19|) . We have the following: 

Theorem 3.2. Let {Q, g) be a compact manifold with connected smooth 
boundary and with nonnegative scalar curvature. Assume that Q is 
simply connected and suppose dfl has positive Gauss curvature and 
positive mean curvature with respect to the outward normal. Suppose 
{^,g) has no horizons. Then m{Q) < rriBYid^l). Equality holds if and 
only if fl is a domain in M.^ . 

Remark 3.3. // dQ has more than one components, then it is easy 
to see that Q will have an outermost minimizing horizon with respect 
to any boundary component because dQ has positive mean curvature. 
If dQ is connected and fl is not simply connected, then fl contains a 
minimal sphere or real projective space by jH] . 

Proof of Theorem VJ.^A Assume that fl has no horizons. Suppose fl 
is not diffeomorphic to an open 3-ball in M^, then fl must contains a 
minimal sphere S or projective plane Since fl is simply connected, 
S must be a minimal sphere and is the boundary of some precompact 
open set i?, see p. 107]. So 5* is a horizon. This is a contradiction. 
Hence fl is diffeomorphic to an open 3-ball in . 

Let f2i CC ^2 C with smooth boundaries. Let ii^ be a connected 
precompact minimizing hull in fl2 with boundary such that E <zfli. 
We want to prove that m by {dfl) > a^^-^^^mH{dE). Again, we may 
assume that mnidE) > 0. 

Let E' be the strictly minimizing hull of E in fl. E' exists because dfl 
has positive mean curvature. Moreover, E' has C^'^ boundary because 
E has boundary. Note that E' and dE' are connected because fl is 



16 



Yuguang Shi and Luen-Fai Tarn 



diffeomorphic to the unit ball in M.^. By Theorem 
rriBYidn) > mnidE') 

- V 167r V 167r Jqe J 

because dE' \ dE is minimal and the mean curvatures of dE and dE' 
are equal a.e. on their common part. Suppose E' CC ^2, then \dE'\ > 
\dE\ and we have mBY{dQ) > mnidE). 

Suppose E' is not a precompact set in Q2, then dE' fl (M \ Q2) 7^ 0- 
On the other hand, if dE' fl dE = then dE' would be a horizon 
contradicting the assumption that Q contains no horizons. Hence 

dE' nniD dE' ndE ^ Us 

and there is a point x G dE' and d{x, d^2) = f ? where d is the distance 
between Qi and dQ2- By Lemma f2. 31 we have 

(3.13) \dE'\>al^.^^Jdn^\>al.^^JdE\ 

because E is a minimizing hull in Q2 ■ Hence by fl3.12|l , ()3.13j) we have 

(3.14) niBYidQ) > an,;u,mH{dE). 

This completes the proof of the first part. If equality holds, then from 
the proof above, for any e > we can find a strictly minimizing hull 
E' in Q with C^'^ boundary such that mBvidQ) < mnidE') +e. From 
the proof of Theorem 13. H one can conclude that Vt must be a domain 
in M?. □ 

Let VL be as in the theorem. Isometrically embed dVt in R'^. Let R 
be the radius of the smallest circumscribed ball of dVt in 'E?. We have 
the following: 

Corollary 3.4. Let (f2, g) he a compact manifold with nonnegative 
scalar curvature and with connected boundary which has positive mean 
curvature and positive Gauss curvature. Suppose Vl is simply connected 
and suppose m{VL) > mBY{dQ), then there is a horizon in Q unless Q 
is a domain in M.^ . Hence if m{Q) > 2R, then Q contains a horizon. 
In particular, if m{fl) > 2diam{dQ) then Q contains a horizon. Here 
diam{dQ) is the diameter of dQ with respect to the metric induced by 
9- 

Proof. The first statement is just a restatement of Theorem 13.21 Since 

(3.15) mBY{n)<—l Ho < 2R < 2diam{dn) 

8^ Jan 
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by the Minkowski integral formula p. 136], the fact that dfl has 
positive Gauss curvature and the Gauss-Bonnet formula. □ 

There are examples so that m{Q) > rriBYi^)- Consider the metrics 
(]R^,(is^ = u'^{dp'^ + p^da"^)) defined in the proof of Proposition 12.11 
where dp^ + p^da"^ is the Euclidean metric. Since as m 0, ds^ 
converges uniformly on the Euclidean ball i?4 = {x G |x| < 4} 
to the standard metric on the unit sphere, we see that for sufficiently 
small m > 0, the boundary of B^- is mean convex for all < r < 2 
with respect to dsl^. Bi is a minimizing hull in B2. Moreover, there 
is 5 > such that the Hawking mass mnidBi) of dBi with respect to 
ds'^ is at least 6, provided m > is small enough. Hence for all m > 
small enough, afi-^.Q^^i^Bi, B2) > 5. Now consider the domain Vt = Bp, 
where p = 8/m > 2po- It is easy to see that dQ has positive mean 
curvature and positive Gauss curvature with diameter d < Cm, here 
C is a universal constant. Hence 

m{n) > an^.n^m{Bi] B2) > S > 2d > mBy(^]) 

provided m is small enough. 

There is another sufficient condition for the existence of horizons in 
terms of the Hawking mass of other kind of surfaces. Suppose E <ZG Q 
is a bounded subset in Q such that E and connected and the volume of 
E is equal vo- Using the terminology in [7], dE is called a round sphere 
if dE is smooth and connected and has least area among all subsets of 
Q with locally finite perimeter with volume vq. The volume of a set of 
locally finite perimeter is simply the measure of the set with respect to 
the volume element of Q. If the scalar curvature of Q is nonnegative, 
and dE is a round sphere, then the Hawking mass mnidE) of dE is 
nonnegative by [7[. We have the following: 

Theorem 3.5. Let [fl, g) be a compact Riemannian three manifold 
with smooth and connected boundary and with nonnegative scalar cur- 
vature. Assume that VL is simply connected and suppose that both the 
Gauss curvature and mean the curvature of the boundary( with respect 
to outward normal) is positive. If there is a round sphere BE in Vt such 
that 

mnidE) > mBvidVL), 
then there is a horizon in Vt. 

First, we prove the following lemma: 

Lemma 3.6. Let {^l,g) be a compact Riemannian three manifold with 
smooth boundary and with nonnegative scalar curvature such that dQ 
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has positive mean curvature. Suppose E CC ^ such that dE is a round 
sphere. Then either Vl contains a horizon or E is a minimizing hull in 

n. 

Proof. Suppose E is not minimizing hull in Q. Since dQ has positive 
mean curvature, there exists a strictly minimizing hull E' containing 
E with E' (ZG Q. Since E is not a minimizing hull, \dE'\ < \dE\. Let 
vq be the volume of E, then vol{E') > vq because E is a. round sphere. 
Choose eo > small enough so that dQ^ is smooth and has positive 
mean curvature for all < e < 2eo, where = {x & Q\ d{x, dQ) > e}. 
Moreover, eo is chosen so that £" CC Qeo- Let $ be the family of 
precompact subsets F with locally finite perimeter of with the 
properties that vol{F) > fo, and \dF\ < l{\dE'\ + \dE\). $ is not 
empty because £" G Let 

(3.16) vi = inf vol(F). 

Note that v\ > fo. We claim that v\ > vq. In fact, let Fj G $ be such 
that \\va.i^^vol[F^ = Vi. By compactness theorem for sets of locally 
finite perimeter, we may assume that the characteristic functions of 
Fi converges in L^{Q) to a the characteristic function of a subset F 
of fl^g with locally finite perimeter such that vol{F) = Vi. By lower 
semi continuity, \dF\ < ^{\dE'\ + \dE\) < \dE\. (For simphcity, we 
use \dF\ to denote the area of the reduced boundary of F). Hence 
f 1 = vol{F) > vq by the definition of round spheres. 
Let 

(3.17) a = inf \dF\. 

Choose a sequence Fj G $ such that limj^oo \dFi\ = a. Since Vi > Vq 
and \dFi\ < ^{\dE'\ + \dE\), by the approximation result of sets of 
locally finite perimeter, see ^U], we may choose Fi such that dFi is 
smooth. Choose 2eo > ei > eo such that vol{fl^g \ fi^i) < ~ "^o)- 
Consider the set Fi = Fi f] a^, then \dFi\ < \dFi\ < l{\dE'\ + \dE\) 
since dfl^ has positive mean curvature for all < e < 2eo. Moreover, 

(3.18) vol{Fi) > vol{Fi) - vol{n,, \ aj > ^{vi + vo) > vq 

by ()3.1(jj) . Hence Fj G $ and limj^oo \dFi\ = a. Passing to a subse- 
quence if necessary, we may assume that the characteristic functions of 
Fi converges in (Q) to a the characteristic function of a subset F of 
fi^i with locally finite perimeter. Moreover, vol{F) > \{vi + Vq) > Vq 
by (jTTSjl and \dF\ = a. Since F' G $, a < \dE'\ < |(|5F'| + \dE\). 
Hence F G 
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Let Bx{r) CC ^leo with vol{Bx{r)) < — Vq). Suppose F is 
any set of locally finite perimeter such that the symmetric difference 
FAF CC 5^(r), then F CC fi^o, 

vol{F) > vol{F) - vol{B,{r)) > ^{vi + vo) > vq 

by ^M- If l^^l < l^^l then F G $ because \dF\ < li\dE'\ + \dE\). 
This is impossible by the construction of F. Hence one can see that F 
is minimizing in any geodesic ball inside with radius small enough 
and F is a compact minimal surface. That is to say fl contains a 
horizon. □ 

Proof of Theorem \3.5\ Let BE be a round sphere. E is not a mini- 
mizing hull in by Theorem 13 . II and the assumption that mH{dE) > 
mBY{d^)- By Lemma we see that Q contains a horizon. This 
completes the proof of the theorem. □ 

4. On the Bartnik quasi-local mass 

In this section, we will discuss relation between m{Q) and the quasi- 
local mass introduced by Bartnik p. We will also discuss some prop- 
erties of the Bartnik quasi-local mass. We use the definition of the 
Bartnik quasi-local mass mB{^) in |2| of a compact manifold {^,g) 
with smooth boundary and with nonnegative scalar curvature. 

Let VM.O be the set of complete noncompact AF manifolds with 
nonnegative scalar curvature which admit no minimal two spheres or 
projective planes. For M G VM.O-, ^ CC M means that VL is isometri- 
cally embedded in M. Then the Bartnik quasi- local mass mB{^) of VL 
is defined as: 

(4.1) niBiVL) = inf {mADM{M)\ VL M} 

AleVMo 

If Q GG M d VM.01 then M is said to be an admissible extension of 
VL. We should remark that for M G VAAo-, M is topologically R^, by 
mj, see also 0. 
We have the following lower bound of mB{^)- 

Theorem 4.1. Suppose Vt has an admissible extension. Then 

(4.2) ruBiP) > m{n). 

Proof. The proof is exactly the same as the proof of Theorem 13.11 □ 

It was proved in [T^ that if 771^(^2) = 0, then VL is locally flat. Hence 
it is a domain in 'E? if dVL is topologically a sphere. If the boundary 
of Vt has several components, then one can show that if 771^(^2) = 0, 
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then Q is still a domain in M.^ provided that each component of dfl is 
topologically a sphere. This follows from the following lemma. 

Lemma 4.2. Let {Q, g) be a compact three manifold with boundary with 
nonnegative scalar curvature such that Q has an admissible extension. 
Suppose mB{^) = 0. Let Si,...,Sfc be the boundary components of 
Q. Then Q can be extended to a manifold Q with compact closure 
with nonnegative scalar curvature such that Q has only one boundary 
component which is equal to Sj for some i and msi^) = 0. 

Proof. Let M; be a minimizing sequence of admissible extensions of Q 
for mB(f2). Then each Mi is diffeomorphic to M^. Each Hj separates 
into two components. Hence by taking a subsequence if necessary, 
we may assume that Q is in the interior of Si, say, for all /. Let Q 
be the interior of Ei in Mi. Note that fl is diffeomorphic to the unit 
ball in M^. Then O C fi, the scalar curvature of Q is nonnegative, and 
dfl = Si. We want to prove that 771^(0) = 0. 

Let Ml be the manifold obtained in the following way: Mi = Mi on 
M/\interior of Si and replace by in the interior of Si, so that in a 
neighborhood of Si, Mi = Mi. We want to prove that if I is large. Mi 
will not contain any stable minimal spheres. 

Choose a e > be small enough, such that {x G Q\ d{x, Si) = e} is 
also topologically a sphere and so that if U = {x E fl\ d{x, Si) < e}, 
then [/ n S j = for all j 7^ 1 . Let be an outermost horizon of Mi . If 
Nn{Q\U) = 0, then C Mi. This is impossible by the fact that Mi is 
admissible and the fact that each component of is a stable minimal 
sphere, see |12| E] . Hence iV fl (f2 \ [/) 7^ and there is a constant c > 
independent of i, such that |A^| > c > by Lemma [2.31 On the other 
hand, 

lim mADAiiMi) = lim mADM{Mi) = 0. 

/— >oo Z— >oo 

Hence by the Penrose inequality in [T^l^. no such exists if I is large 
enough. 

Note that Mi is homeomorphic to M^. Hence Mi contains no minimal 
projective plane. If Mi contains a minimal sphere S, then strictly 
minimizing hull of the interior of 5* is an outermost horizon. Hence no 
such S exists. We conclude that Mi is an admissible extension of fl 
and so mB{^) = 0. □ 

From this, we obtain the following: 

Proposition 4.3. Let {^,g) be as in the lemma. Suppose 771^(1]) = 
and each Sj is topologically a sphere, then fl is a domain in M'^. 
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In case the boundary of dQ is not a sphere, it is still unknown if 
ms(r2) = will imply is a domain in M^. In the following, using 
Theorem 14. II we are going to construct a locally flat compact manifold 
which either has no admissible extension or its Bartnik mass is positive. 

Example 4.4. Consider the cylinder in M^; 

C = {{x, y, z) e R^\x^ + y^ <r^,0 < z < I}. 

Let T be the solid torus by identifying {x,y,0) and {x,y,l) in C. If I 
is small enough depending only on r, then the Hawking mass of T is 
positive. Moreover, ifT has an admissible extension, then the Bartnik 
mass is also positive. 

To prove that mniT) > 0, it is easy to see that the mean curvature 
H of dC is -, and 




Suppose / < 8r, hence 

[ < 16n. 

JdC 

We have mjy(T) > 0. 

Now suppose that T has an admissible extension, we may conclude 
that msiT) > because of the following consequence of Theorem 14.11 

Proposition 4.5. Suppose {^,g) has an admissible extension so that 
dQ is connected. Suppose dQ has positive mean curvature and suppose 
niHidn) > 0. Then mei^) > 0. 

Proof. Choose eo > be small enough such that dfl^ is smooth with 
positive mean curvature for all < e < eo, where 

n, = {xe n\ d{x,dn) > e}. 

Moreover, by choosing eo small enough, we may also assume that 
muidVL^) > for all < e < eo. Then each is a minimizing hull in Q 
because Q \ fl^^ is foliated by positive mean curvature surfaces. Hence 
m{Q^; Q) > provided e > is small enough. In particular, m{fl) > 0. 
By Theorem 14. ![ the proposition follows. □ 

Even though it is unclear if the solid torus T in Example 14.41 has an 
admissible extension. One can prove that T has no smooth static exten- 
sion. Namely, there is no AF manifold (M, g) with one end satisfying 
the following: 

(a) g is smooth. 

(b) T is isometrically embedded in M. 
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(c) There is a smooth function u defined on M\interior of T such 
that n ^ 1 at infinity, Ric = u^^V^u and Au = 0. 

In fact, if (M, g) is such an extension, then by jTH|, m = 1 — ^ + 0(r^^) 
where r is the Euchdean distance in a coordinate system near infinity in 
the definition of AF manifold and m is the ADM mass of M. Moreover, 
let E = dT, then on E (see H): 

(4.3) A^u + H^ = 

where As is the Laplacian on S, u is the unit outward normal and 
H > is the mean curvature of S. Here we have used the fact that S 
is fiat, H is constant and T is locally fiat. Hence we have 

= / Au 

J M\T 

f du 

= Anm — / — 
is du 

= Anm + j A^u 

Hence M must be M^. This is impossible by the uniqueness of embed- 
ding. 
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